Prompt Generator for Stable Diffusion
Contents
Prompt Generator for Stable Diffusion#
Note
Install ekorpkit package first.
Set logging level to Warning, if you don’t want to see verbose logging.
If you run this notebook in Colab, set Hardware accelerator to GPU.
!pip install ekorpkit[art] exit()
Preparing the environment#
%load_ext autotime
%load_ext autoreload
%autoreload 2 # reload modules before executing user code, set to 0 to disable
from ekorpkit import eKonf
eKonf.setLogger("INFO")
eKonf.set_cuda(device=4)
print("version:", eKonf.__version__)
is_notebook = eKonf.is_notebook()
is_colab = eKonf.is_colab()
print("is notebook?", is_notebook)
print("is colab?", is_colab)
if is_colab:
eKonf.mount_google_drive()
eKonf.set_workspace(workspace="/content/drive/MyDrive/workspace/", project="ekorpkit-book")
INFO:ekorpkit.base:Setting cuda device to ['A100-SXM4-40GB (id:4)']
INFO:ekorpkit.utils.notebook:shell type: ZMQInteractiveShell
INFO:ekorpkit.utils.notebook:Google Colab not detected.
INFO:ekorpkit.base:Setting EKORPKIT_WORKSPACE_ROOT to /content/drive/MyDrive/workspace/
INFO:ekorpkit.base:Setting EKORPKIT_PROJECT to ekorpkit-book
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env
version: 0.1.40.post0.dev3
is notebook? True
is colab? False
('/content/drive/MyDrive/workspace', 'ekorpkit-book')
time: 2.29 s (started: 2022-11-02 08:08:26 +00:00)
Preparing the dataset#
from ekorpkit.models.art.prompt import PromptGenerator
pgen = PromptGenerator(verbose=False)
2022-11-02 08:08:36.692394: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
INFO:ekorpkit.batch:Batch name: prompt-batch, Batch num: 14
time: 3.93 s (started: 2022-11-02 08:08:35 +00:00)
prompts = pgen.generate_prompts(prompt="island")
prompts
INFO:ekorpkit.batch:Using existing path: /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen
INFO:ekorpkit.batch:Batch name: prompt-batch, Batch num: 14
INFO:ekorpkit.batch:Merging config with args: {}
INFO:ekorpkit.batch:Setting seed to 1075645782
INFO:ekorpkit.models.art.prompt:Loading model from /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts
INFO:ekorpkit.batch:Saving config to /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/stable-prompt/prompt-batch/prompt-batch(14)_config.yaml
['islanders, hyperdetailed, digital painting, concept art, sharp focus, illustration, fantasy world, photorealistic, masterpiece a portrait of a woman in a sci - fi outfit with cybernetic implants, cyberpunk tech wear',
'island, jossi riabovitchev, marc simonetti, yoshitaka amano 3 d goddess close - up profile, symmetrical perfect face, flowing long hair, dazzling light orbs, highly',
'island 2 4 5 3 concept art oil painting of a lushly lit cityscape with lush vegetation and stunning voluptuous ethereal pink pink neon flowers, stunning voluptuous nordic imagery, voluptuous oriental art nouve',
'island, volumetric lighting, red and cyan theme, artstation, concept art by craig mullins dnd character portrait concept art, dnd, rpg character, trending on artstation, digital art, by studio',
'island, concept art, matte painting, fantasy, sharp focus, vibrant colors A full portrait of a beautiful ethereal kat dennings by Studio Ghibli, Makoto Shinkai, by Artgerm, by W',
'island - kalachakra, full body shot, white hair, red hair, brown eyes, white narrow mid cut shirt, jeans, large bellybutton, high waist, dramatic light, cinematic wallpaper, anime, ilya kuvshin',
'islandland, new research notes, post grunge, concept art by peter elson johannes voss molly millions, dark fantasy magic, concept art, trending on art station, award-winning photograph, masterpiece',
'island on fire in las vegas, epic landscape, detailed landscape, arctic, fantasy magic, fairytale, storyteller, atmospheric, intricate, headshot, highly detailed, digital painting, artstation, concept art, sharp focus',
'island, concept art, sharp focus, illustration, art by artgerm and greg rutkowski and alphonse mucha, footage from space camera',
'island, artstation, cgsociety, concept art A group portrait of cute Incredibly beautiful cute Blackpink Lalisa Manoban wearing fantastic dress in the style of Assamese bihu mekhela sad']
time: 18.6 s (started: 2022-11-02 08:08:41 +00:00)
pgen.train(model_name="ekorpkit/stable-prompts-2")
WARNING:datasets.builder:Using custom data configuration Gustavosta--Stable-Diffusion-Prompts-d22aeec0ba2a9fdb
WARNING:datasets.builder:Reusing dataset parquet (/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/cache/Gustavosta___parquet/Gustavosta--Stable-Diffusion-Prompts-d22aeec0ba2a9fdb/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)
Token indices sequence length is longer than the specified maximum sequence length for this model (2613888 > 1024). Running this sequence through the model will result in indexing errors
***** Running training *****
Num examples = 18847
Num Epochs = 1
Instantaneous batch size per device = 1
Total train batch size (w. parallel, distributed & accumulation) = 8
Gradient Accumulation steps = 1
Total optimization steps = 2356
Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
ERROR:wandb.jupyter:Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.
wandb: Currently logged in as: entelecheia. Use `wandb login --relogin` to force relogin
wandb version 0.13.4 is available! To upgrade, please run:
$ pip install wandb --upgrade
Tracking run with wandb version 0.13.3
Run data is saved locally in
/workspace/projects/ekorpkit-book/ekorpkit-book/docs/lectures/aiart/wandb/run-20221102_081028-3heixjgo
[2356/2356 04:59, Epoch 1/1]
| Step | Training Loss |
|---|---|
| 200 | 3.776300 |
| 400 | 2.705100 |
| 600 | 2.517600 |
| 800 | 2.376600 |
| 1000 | 2.295400 |
| 1200 | 2.231800 |
| 1400 | 2.175900 |
| 1600 | 2.143700 |
| 1800 | 2.104900 |
| 2000 | 2.082700 |
| 2200 | 2.081700 |
Training completed. Do not forget to share your model on huggingface.co/models =)
Waiting for W&B process to finish... (success).
Run history:
| train/epoch | ▁▂▂▃▄▄▅▆▆▇▇█ |
| train/global_step | ▁▂▂▃▄▄▅▆▆▇▇█ |
| train/learning_rate | █▇▇▆▅▄▄▃▂▂▁ |
| train/loss | █▄▃▂▂▂▁▁▁▁▁ |
| train/total_flos | ▁ |
| train/train_loss | ▁ |
| train/train_runtime | ▁ |
| train/train_samples_per_second | ▁ |
| train/train_steps_per_second | ▁ |
Run summary:
| train/epoch | 1.0 |
| train/global_step | 2356 |
| train/learning_rate | 0.0 |
| train/loss | 2.0817 |
| train/total_flos | 4924659858407424.0 |
| train/train_loss | 2.38621 |
| train/train_runtime | 353.5289 |
| train/train_samples_per_second | 53.311 |
| train/train_steps_per_second | 6.664 |
Synced ekorpkit/stable-prompts-2: https://wandb.ai/entelecheia/ekorpkit-book-prompt-generator/runs/3heixjgo
Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
Find logs at:
./wandb/run-20221102_081028-3heixjgo/logsINFO:ekorpkit.models.art.prompt:Saving model to /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2
Configuration saved in /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/config.json
Model weights saved in /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/pytorch_model.bin
INFO:ekorpkit.batch:Saving config to /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/stable-prompt/prompt-batch/prompt-batch(14)_config.yaml
time: 7min 9s (started: 2022-11-02 08:09:17 +00:00)
pgen.load_model(model_name="ekorpkit/stable-prompts-2")
INFO:ekorpkit.models.art.prompt:Loading model from /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2
loading configuration file /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/config.json
Model config GPT2Config {
"_name_or_path": "/content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2",
"_num_labels": 1,
"activation_function": "gelu_new",
"architectures": [
"GPT2LMHeadModel"
],
"attn_pdrop": 0.1,
"bos_token_id": 50256,
"embd_pdrop": 0.1,
"eos_token_id": 50256,
"id2label": {
"0": "LABEL_0"
},
"initializer_range": 0.02,
"label2id": {
"LABEL_0": 0
},
"layer_norm_epsilon": 1e-05,
"model_type": "gpt2",
"n_ctx": 1024,
"n_embd": 768,
"n_head": 12,
"n_inner": null,
"n_layer": 6,
"n_positions": 1024,
"reorder_and_upcast_attn": false,
"resid_pdrop": 0.1,
"scale_attn_by_inverse_layer_idx": false,
"scale_attn_weights": true,
"summary_activation": null,
"summary_first_dropout": 0.1,
"summary_proj_to_labels": true,
"summary_type": "cls_index",
"summary_use_proj": true,
"task_specific_params": {
"text-generation": {
"do_sample": true,
"max_length": 50
}
},
"torch_dtype": "float32",
"transformers_version": "4.21.2",
"use_cache": true,
"vocab_size": 50260
}
loading weights file /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2/pytorch_model.bin
All model checkpoint weights were used when initializing GPT2LMHeadModel.
All the weights of GPT2LMHeadModel were initialized from the model checkpoint at /content/drive/MyDrive/workspace/projects/ekorpkit-book/disco-imagen/models/ekorpkit/stable-prompts-2.
If your task is similar to the task the model of the checkpoint was trained on, you can already use GPT2LMHeadModel for predictions without further training.
time: 1.42 s (started: 2022-11-02 08:04:02 +00:00)
Generate images#
text_prompts = "a living tiger inside a gorgeous cave in a futuristic Jeju island, relaxing night, watercolour on the ceiling, glowing on glass, light art, dark, dark, gorgeous, dark, bright in an abandoned village in a futuristic alien nightscape"
batch_name = "cave-jeju"